Skip to content

feat(editor): hide the camera during silence - #3626

Draft
AatmanAJ wants to merge 6 commits into
CapSoftware:mainfrom
AatmanAJ:feat/silence-camera
Draft

AatmanAJ wants to merge 6 commits into
CapSoftware:mainfrom
AatmanAJ:feat/silence-camera

Conversation

@AatmanAJ

@AatmanAJ AatmanAJ commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Adds "Hide When Silent" to the Camera tab: the camera fades away when you stop talking and comes back just before you speak again.

How it works

  • Detection. cap_audio::detect_speech_in_file runs RNNoise's voice probability over each clip's mic, with a -55 dBFS floor. It runs in the background once per opened project, and a slider drag reuses the result.
  • Output. Silences become HideCamera scene segments tagged fromSilence. They are normal scenes, so they appear on the Scene track, can be trimmed or deleted, and follow ripple cuts. The renderer and export need no changes.
  • Timing. The camera leaves after the "Silence Before Hiding" delay (1 to 10 seconds, default 2) and returns 0.3 seconds before speech. Silences too short to fit both fades are skipped, and blips under 0.2 seconds don't bring the camera back.
  • Time mapping. TimelineConfiguration::clip_source_windows maps speech through cuts, clip speed, transitions, fullscreen text holds, and mic offsets including timing repairs.
  • Manual scenes win. Regenerating only replaces tagged segments, and generated ones keep a margin around hand-made scenes. Changing a generated segment's mode clears its tag, so a user's edit survives the next regenerate.
  • Failure direction. A clip with no mic, or a mic that can't be read, counts as speech. A bad read can only keep the camera on screen, never hide it.

The toggle only shows for recordings with a microphone. If nothing qualifies, it shows a notice and stays off. Removing the Scene track also turns the setting off, and presets leave it alone because it belongs to the timeline.

Why segments, not a render-time rule

Cap's other automation, like auto zoom and auto 3D shots, writes editable segments. Doing the same here means no per-frame merge in the renderer, no analysis step before export, and every hidden range is visible and fixable on the timeline. The trade-off is that adding or replacing footage later doesn't regenerate the segments. Toggling off and on, or moving the slider, refreshes them.

Testing

  • cargo test -p cap-audio -p cap-project -p cap-rendering --lib filtered to silence and speech: all pass, covering windowing, cut and speed mapping, offsets, manual scene margins, fade timing through InterpolatedScene, regenerate behavior, and unreadable mics.
  • cargo test --bin cap-gpui editor_ passes in apps/desktop-gpui (482 tests), including the i18n surface tests for the new strings. cargo check --workspace --all-targets passes.
  • Ran the detector on a real 10.7-minute mic recording in a debug build: 5.2 seconds, 320 speech spans.

Ran in the real app on a project with a camera and an 8-second silence: the toggle generated a Hide Camera scene that lines up with the gap, and after reopening, moving the delay rebuilt it and saved both together.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue was established.

Summary

Adds “Hide When Silent” to the Camera tab. The editor analyzes microphone speech and creates editable HideCamera scenes.

  • The editor checks microphone clips for speech in the background.
  • Long quiet stretches create camera-hide scenes that follow the edited timeline.
  • The Camera tab lets people turn silence-based hiding on and set its delay.

Reviews (10) · Last reviewed commit: "test(editor): mark the drag-transition s..." · Reviewed by Greptile

@vercel

vercel Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

Comment thread crates/rendering/src/silence_scenes.rs Resolved
@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

3 similar comments
@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ
AatmanAJ force-pushed the feat/silence-camera branch from d095d05 to be8bb29 Compare November 1, 2026 13:53
@AatmanAJ

AatmanAJ commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

@AatmanAJ
AatmanAJ force-pushed the feat/silence-camera branch from be8bb29 to aa0549d Compare November 4, 2026 18:07
@AatmanAJ

AatmanAJ commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

Aatman Jain and others added 6 commits November 4, 2026 22:50
Detect speech on each clip's mic and generate Hide Camera scene segments
for the long silences. The segments are ordinary scenes, so they show on
the Scene track, follow cuts, and export without any renderer changes.
Analyze speech when the delay moves on a reopened project, let clearing the
Scene track cancel a pending result, and count speech from both clips
during a transition.
Hold the delay until speech is ready so undo restores both together, and
stop treating an all-unreadable analysis as a usable cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AatmanAJ
AatmanAJ force-pushed the feat/silence-camera branch from aa0549d to 39166e5 Compare November 4, 2026 23:21

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant